<!DOCTYPE html>
<html lang="zh">
    <head>
        <meta charset="utf-8">
        <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
        <title>cs229 第八节 | Dirac Lee</title><meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="robots" content="noodp" />
<meta name="Description" content="cs229 第八节"><link rel="prev" href="https://diraclee.gitee.io/2020/10/cs229_lecture7/" /><link rel="next" href="https://diraclee.gitee.io/2020/10/sklearn-%E5%B8%B8%E7%94%A8%E6%A8%A1%E5%9E%8B%E5%8F%82%E6%95%B0/" /><link rel="canonical" href="https://diraclee.gitee.io/2020/10/cs229_lecture8/" />
<link rel="shortcut icon" type="image/x-icon" href="/favicon.ico" />
<link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon.png">
<link rel="icon" type="image/png" sizes="32x32" href="/favicon-32x32.png">
<link rel="icon" type="image/png" sizes="16x16" href="/favicon-16x16.png">
<link rel="manifest" href="/site.webmanifest">
<link rel="mask-icon" href="/safari-pinned-tab.svg" color="#5bbad5">
<meta name="msapplication-TileColor" content="#da532c">
<meta name="theme-color" content="#ffffff"><meta property="og:title" content="cs229 第八节" />
<meta property="og:description" content="cs229 第八节" />
<meta property="og:type" content="article" />
<meta property="og:url" content="https://diraclee.gitee.io/2020/10/cs229_lecture8/" />
<meta property="article:published_time" content="2020-10-17T00:00:00+00:00" />
<meta property="article:modified_time" content="2020-10-17T00:00:00+00:00" />
<script type="application/ld+json">
    {
        "@context": "http://schema.org",
        "@type": "BlogPosting",
        "headline": "cs229 第八节",
        "mainEntityOfPage": {
            "@type": "WebPage",
            "@id": "https:\/\/diraclee.gitee.io\/2020\/10\/cs229_lecture8\/"
        },"image": {
                "@type": "ImageObject",
                "url": "https:\/\/diraclee.gitee.io\/logo.png",
                "width":  800 ,
                "height":  600 
            },"genre": "posts","keywords": "机器学习, cs229","wordcount":  1519 ,
        "url": "https:\/\/diraclee.gitee.io\/2020\/10\/cs229_lecture8\/","datePublished": "2020-10-17T00:00:00\x2b00:00","dateModified": "2020-10-17T00:00:00\x2b00:00","license": "This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.","publisher": {
                "@type": "Organization",
                "name": "xxxx",
                "logo": {
                "@type": "ImageObject",
                "url": "https:\/\/diraclee.gitee.io\/logo.png",
                "width":  127 ,
                "height":  40 
                }
            },"description": "cs229 第八节"
    }
    </script><link rel="stylesheet" href="/css/style.min.css"><link rel="stylesheet" href="/css/lib/fontawesome-free/all.min.css"><link rel="stylesheet" href="/css/lib/animate/animate.min.css"></head>
    <body><script>
            window.isDark = (window.localStorage && window.localStorage.getItem('theme')) === 'dark';
            window.isDark && document.body.classList.add('dark-theme');
        </script><div class="wrapper"><nav class="navbar">
    <div class="navbar-container">
        <div class="navbar-header animated bounceIn">
            <a href="https://diraclee.gitee.io">Dirac Lee</a>
        </div>
        <div class="navbar-menu"><a class="menu-item" href="https://diraclee.gitee.io/posts" title="">文章</a><a class="menu-item" href="https://diraclee.gitee.io/tags" title="">标签</a><a class="menu-item" href="https://diraclee.gitee.io/categories" title="">分类</a><a class="menu-item" href="https://diraclee.gitee.io/about" title="">关于</a><a class="menu-item" href="https://diraclee.gitee.io" title="English"><i class="fas fa-language fa-fw"></i></a><a href="javascript:void(0);" class="theme-switch"><i class="fas fa-adjust fa-rotate-180 fa-fw" title="切换主题"></i></a>
        </div>
    </div>
</nav><nav class="navbar-mobile">
    <div class="navbar-container">
        <div class="navbar-header">
            <div class="navbar-header-title animated bounceIn">
                <a href="https://diraclee.gitee.io">Dirac Lee</a>
            </div>
            <div class="menu-toggle" id="menu-toggle">
                <span></span><span></span><span></span>
            </div>
        </div>
        <div class="navbar-menu" id="mobile-menu"><a class="menu-item" href="https://diraclee.gitee.io/posts" title="">文章</a><a class="menu-item" href="https://diraclee.gitee.io/tags" title="">标签</a><a class="menu-item" href="https://diraclee.gitee.io/categories" title="">分类</a><a class="menu-item" href="https://diraclee.gitee.io/about" title="">关于</a><a class="menu-item" href="https://diraclee.gitee.io" title="English"></a><a href="javascript:void(0);" class="theme-switch"><i class="fas fa-adjust fa-rotate-180 fa-fw" title="切换主题"></i></a>
        </div>
    </div>
</nav><main class="main">
                <div class="container"><article class="page"><h1 class="post-title animated flipInX">cs229 第八节</h1><div class="post-meta">
            <div class="post-meta-main"><a class="author" href="https://diraclee.gitee.io" rel="author" target="_blank">
                    <i class="fas fa-user-circle fa-fw"></i>Dirac Lee
                </a>&nbsp;<span class="post-category">收录于&nbsp;<i class="far fa-folder fa-fw"></i><a href="https://diraclee.gitee.io/categories/%E5%AD%A6%E4%B9%A0%E7%AC%94%E8%AE%B0/">学习笔记</a>&nbsp;</span></div>
            <div class="post-meta-other"><i class="far fa-calendar-alt fa-fw"></i><time datetime=2020-10-17>2020-10-17</time>&nbsp;
                <i class="fas fa-pencil-alt fa-fw"></i>约 1519 字&nbsp;
                <i class="far fa-clock fa-fw"></i>预计阅读 4 分钟&nbsp;</div>
        </div><div class="post-content"><p>SVM</p>
<ul>
<li>优化问题</li>
<li>代表者理论</li>
<li>核函数</li>
<li>核函数举例</li>
</ul>
<a class="post-dummy-target" id="最优边界分类器-回顾"></a><h2>最优边界分类器 (回顾)</h2>
<p>$$
\begin{cases}
\arg \min_{w, b} ||w||^2  \\ \\ 
s.t. \quad {y^{(i)} ( w^T x^{(i)} + b})  \ge 1 \\ 
\end{cases}
$$</p>
<p>证明</p>
<p>$$
\gamma^{(i)} = \frac { y^{(i)} ( w^T x^{(i)} + b ) } {||w||} 
$$</p>
<p>$$
\gamma = \min_{i=1}^m \gamma^{(i)}
\implies
\forall \gamma^{(i)} \ge \gamma
$$</p>
<p>优化目标</p>
<p>$$
\begin{cases}
\arg \max_{\gamma, w, b} \gamma  \\ \\ 
s.t. \underbrace{ \frac {y^{(i)} ( w^T x^{(i)} + b} {||w||} \ge \gamma }_{\text{ 每个样例的几何边界大于等于γ }}
\end{cases}
$$</p>
<p>令 $||w|| = \frac 1 \gamma$ ，则优化目标转化为</p>
<p>$$
\begin{cases}
\arg \max \frac 1 {|| w ||}  \\ \\ 
s.t. \quad {y^{(i)} ( w^T x^{(i)} + b} ) \gamma \ge \gamma 
\end{cases}
\iff 
\begin{cases}
\arg \min \frac 1 2 ||w||^2  \\ \\ 
s.t. \quad {y^{(i)} ( w^T x^{(i)} + b})  \ge 1 
\end{cases}
$$</p>
<p>$x^{(i)} \in \R^{100}$</p>
<p>假设 $w = \sum_{i=1}^m \alpha_i y^{(i)} x^{(i)} $</p>
<p>为什么？</p>
<p>直觉1：逻辑回归的启示</p>
<ul>
<li>初始化</li>
</ul>
<p>$$
\theta \coloneqq 0
$$</p>
<ul>
<li>
<p>梯度下降</p>
<ul>
<li>随机梯度下降</li>
</ul>
<p>$$
\theta \coloneqq \theta - \alpha (h_\theta (x^{(i)}) - y^{(i)} ) x^{(i)}
$$</p>
<ul>
<li>批量梯度下降</li>
</ul>
<p>$$
\theta \coloneqq \theta - \alpha \sum_{i=1}^m (h_\theta (x^{(i)}) - y^{(i)} ) x^{(i)}
$$</p>
</li>
</ul>
<p>直觉2：</p>
<p><figure><img src="/svg/loading.min.svg" data-sizes="auto" data-src="https://gitee.com/DiracLee/picbed/raw/master/img/20201019101504.png" alt="" class="lazyload"></figure></p>
<p>$$
w = \sum_{i=1}^m \alpha_i y^{(i)} x^{(i)} 
$$</p>
<p>$$
\begin{cases}
\arg \min \frac 1 2 ||w||^2  \\ \\ 
s.t. \quad y^{(i)} ( w^T x^{(i)} + b)  \ge 1 
\end{cases}
$$</p>
<p>而</p>
<p>$$
\begin{aligned}
\arg \min \frac 1 2 ||w||^2 
&amp;= \arg \min \frac 1 2 ( \sum_{i=1}^m \alpha_i y^{(i)} x^{(i)} )^T ( \sum_{j=1}^m \alpha_j y^{(j)} x^{(j)} ) \\ 
&amp;= \arg \min \frac 1 2 \sum_{i=1}^m \sum_{j=1}^m \alpha_i \alpha_j y^{(i)} y^{(j)} \underbrace{ x^{(i)^T} x^{(j)} }_{内积  &lt; x^{(i)} , x^{(j)} &gt; }
\end{aligned} 
$$</p>
<p>$$
\begin{aligned}
&amp;y^{(i)} ( w^T x^{(i)} + b) \ge 1  \\ 
\implies \quad &amp;y^{(i)} ( (\sum_{j=1}^m \alpha_j y^{(j)} x^{(j)})^T x^{(i)} + b) \ge 1  \\ 
\implies \quad &amp;y^{(i)} ( (\sum_{j=1}^m \alpha_j y^{(j)} &lt; x^{(j)} , x^{(i)} &gt; + b) \ge 1 
\end{aligned}
$$</p>
<p>因此，优化问题变为</p>
<p>$$
\begin{cases}
\arg \min \frac 1 2 \sum_{i=1}^m \sum_{j=1}^m \alpha_i \alpha_j y^{(i)} y^{(j)} &lt; x^{(i)} , x^{(j)} &gt; \\ \\ 
s.t. \quad y^{(i)} ( (\sum_{j=1}^m \alpha_j y^{(j)} &lt; x^{(j)} , x^{(i)} &gt; + b) \ge 1 
\end{cases}
$$</p>
<p>其对偶优化问题为</p>
<p>$$
\begin{cases}
\arg \max \alpha_i - \frac 1 2 \sum_{i=1}^m \sum_{j=1}^m \alpha_i \alpha_j y^{(i)} y^{(j)} &lt; x^{(i)} , x^{(j)} &gt; \\ \\ 
s.t. \quad \alpha_i \ge 0 , \quad \sum_i y^{(i)} \alpha_i = 0
\end{cases}
$$</p>
<ol>
<li>
<p>求解 $\alpha_i$  的参数</p>
</li>
<li>
<p>作出预测</p>
</li>
</ol>
<p>$$
\begin{aligned}
h_{w, b}(x) 
&amp;= g(w^T x + b) \\ 
&amp;= g((\sum_{i=1}^m \alpha_i y^{(i)} x^{(i)})^T x + b) \\ 
&amp;= g(\sum_{i=1}^m \alpha_i y^{(i)} &lt; x^{(i)}, x &gt; + b)
\end{aligned}
$$</p>
<p>预测只需要内积！</p>
<a class="post-dummy-target" id="核函数"></a><h2>核函数</h2>
<ol>
<li>
<p>写出关于 $&lt; x^{(i)}, x^{(j)} &gt;$ 的算法</p>
</li>
<li>
<p>设计 $x \rightarrow \phi(x)$ 的映射</p>
</li>
<li>
<p>设法计算核函数 $K(x, z) = \phi^T(x) \phi(z)$</p>
</li>
<li>
<p>在算法中用核函数 $K(x, z)$ 来替换 $&lt; x, z &gt;$</p>
</li>
</ol>
<p>举例一</p>
<p>$$
x = 
\underbrace {
\begin{bmatrix}
x_1 \\ 
x_2 \\ 
x_3
\end{bmatrix} \in R^n
\implies
\phi(x) = 
\begin{bmatrix}
x_1 x_1 \\ 
x_1 x_2 \\ 
x_1 x_3 \\ 
x_2 x_1 \\ 
x_2 x_2 \\ 
x_2 x_3 \\ 
x_3 x_1 \\ 
x_3 x_2 \\ 
x_3 x_3 \\ 
\end{bmatrix} 
}_{ 需要 O(n^2) 的时间来计算 }
\in \R^{n^2}
$$</p>
<p>$$
\begin{aligned}
K(x, z) 
&amp;=  \phi^T(x) \phi(z) \\ 
&amp;=  (x^T z)^2  \qquad 只需要 O(n) 的时间来计算 
\end{aligned}
$$</p>
<p>证明</p>
<p>$$
\begin{aligned}
\phi(x) = 
\begin{bmatrix}
x_1 x_1 \\ 
x_1 x_2 \\ 
x_1 x_3 \\ 
x_2 x_1 \\ 
x_2 x_2 \\ 
x_2 x_3 \\ 
x_3 x_1 \\ 
x_3 x_2 \\ 
x_3 x_3 \\ 
\end{bmatrix} 
&amp;&amp; 
\phi(z) = 
\begin{bmatrix}
z_1 z_1 \\ 
z_1 z_2 \\ 
z_1 z_3 \\ 
z_2 z_1 \\ 
z_2 z_2 \\ 
z_2 z_3 \\ 
z_3 z_1 \\ 
z_3 z_2 \\ 
z_3 z_3 \\ 
\end{bmatrix} 
\end{aligned}
$$</p>
<p>$$
\begin{aligned}
(x^T z)^2 
&amp;= (x^T z) (x^T z) \\ 
&amp;= (\sum_{i=1}^n x_i z_i ) (\sum_{j=1}^n x_j z_j ) \\ 
&amp;= \sum_{i=1}^n \sum_{j=1}^n (x_i x_j) (z_i z_j) \\ 
&amp;= \phi^T(x) \phi(z)
\end{aligned}
$$</p>
<p>举例二</p>
<p>$$
K(x, z) =  (x^T z + c )^2 = \phi^T(x) \phi(z)
$$</p>
<p>则</p>
<p>$$
x = 
\begin{bmatrix}
x_1 \\ 
x_2 \\ 
x_3
\end{bmatrix} \in R^n
\implies
\phi(x) = 
\begin{bmatrix}
x_1 x_1 \\ 
x_1 x_2 \\ 
x_1 x_3 \\ 
x_2 x_1 \\ 
x_2 x_2 \\ 
x_2 x_3 \\ 
x_3 x_1 \\ 
x_3 x_2 \\ 
x_3 x_3 \\ 
\sqrt{2c} x_1 \\ 
\sqrt{2c} x_2 \\ 
\sqrt{2c} x_3 \\ 
c \\ 
\end{bmatrix} 
\in \R^{n^2 + n + 1}
$$</p>
<p>扩展</p>
<p>$$
K(x, z) =  (x^T z + c )^d
$$</p>
<p>则 $\phi(x)$ 有至多 O(d) 的单项式的所有 $ \begin{pmatrix} n+d \\ d \end{pmatrix} $ 特征 ($\approx (n + d)^d$ 维)</p>
<p>最优边界优化器 + 核函数技巧 = SVM</p>
<p>SVM 可以将特征映射到很高的维度，但计算时只需要 $O(n)$ 的时间复杂度</p>
<p>如何选择核函数？</p>
<ul>
<li>如果 x 和 z 很 “相似”，那么 $K(x, z) = \phi^T(x) \phi(z) $ 的结果应该尽可能 “大”</li>
<li>如果 x 和 z 很 “不相似”，那么 $K(x, z) = \phi^T(x) \phi(z) $ 的结果应该尽可能 “小”</li>
</ul>
<p>高斯核 $K(x, z) = \exp ( - \frac { || x - z ||^2 } {2 \sigma^2} )$ 很好地符合这个要求</p>
<p>一个函数是否能够作为核函数 $\iff$
是否存在 $\phi$ 使得 $K(x, z) = \phi^T(x) \phi(z) $</p>
<p>也等价于该函数满足以下限制</p>
<p>$$
K(x, z) = \phi^T(x) \phi(z) \ge 0 
$$</p>
<p>有一个定理来证明这一点</p>
<p>$ \lbrace x^{(i)} \dots x^{(i)} \rbrace $ 为任意点</p>
<p>$K \in \R^{ d \times d } $ 为核矩阵</p>
<p>其中</p>
<p>$$
K_{i, j} = K(x^{(i)}, x^{(j)})
$$</p>
<p>对于任意向量 $z$，有</p>
<p>$$
\begin{aligned}
z^T K z 
&amp;= \sum_i \sum_j z_i K_{i, j} z_j \\ 
&amp;= \sum_i \sum_j z_i \phi^T(x^{(i)}) \phi(x^{(j)}) z_j \\ 
&amp;= \sum_i \sum_j z_i \sum_k (\phi^T(x^{(i)}))_k (\phi(x^{(j)}))_k z_j \\ 
&amp;= \sum_k \sum_i \sum_j z_i (\phi^T(x^{(i)}))_k (\phi(x^{(j)}))_k z_j \\ 
&amp;= \sum_k ( \sum_i z_i ( \phi(x^{(i)}) )_k )^2 \ge 0
\end{aligned}
$$</p>
<p>因此 $K &gt;= 0$</p>
<p>这也是一个充分条件，即满足这个条件的函数可以作为核函数</p>
<p>Mercer 定理</p>
<p>核函数 $K$ 合法 $\iff$  $\exists \phi 使得  K(x, z) = \phi^T(x) \phi(z) $ $\iff$ $\forall 点 \lbrace x^{(1)} \dots x^{(d)} \rbrace $，相应的核矩阵 $K \ge 0$</p>
<p>一些常见的核函数</p>
<ul>
<li>
<p>线性核函数：$K(x, z) = x^T z, \quad \phi(x) = x $</p>
</li>
<li>
<p>高斯核函数：$K(x, z) = \exp ( - \frac { || x - z ||^2 } {2 \sigma^2} ), \quad \phi(x) \in R^\infin $</p>
</li>
<li>
<p>多项式核函数：$K(x, z) = (x^T z)^d $</p>
</li>
</ul>
<p>带 L1 正则项的软边界 SVM</p>
<p><figure><img src="/svg/loading.min.svg" data-sizes="auto" data-src="https://gitee.com/DiracLee/picbed/raw/master/img/20201019113923.png" alt="" class="lazyload"></figure></p>
<p>$$
\begin{cases}
\arg \max_{w, b, \xi} \frac 1 2 ||w||^2 + c \sum_{i=1}^m \xi_i \\ \\ 
s.t. \quad  \underbrace{ y^{(i)} ( w^T x^{(i)} + b ) }_{ 函数边界 } \ge 1 - \xi_i , \quad \xi_i \ge 0
\end{cases}
$$</p>
<p>对偶优化问题</p>
<p>$$
\begin{cases}
\arg \max \alpha_i - \frac 1 2 \sum_{i=1}^m \sum_{j=1}^m \alpha_i \alpha_j y^{(i)} y^{(j)} &lt; x^{(i)} , x^{(j)} &gt; \\ \\ 
s.t. \quad 0 \le \alpha_i \le c , \quad \sum_i y^{(i)} \alpha_i = 0
\end{cases}
$$</p>
<p>蛋白质序列分类器</p>
<p>A &hellip; Z</p>
<p><code>BAITSIAIBAJTACI ...</code></p>
<p>$\phi(x) = ?$</p>
<p>$$
\phi(x) = 
\begin{bmatrix}
0 \\ 
0 \\ 
0 \\ 
\vdots \\ 
0 \\ 
0 \\ 
\vdots \\ 
2 \\ 
\vdots \\ 
1 \\ 
\vdots \\ 
0 \\ 
\end{bmatrix}
\begin{matrix}
AAAA \\ 
AAAB \\ 
AAAC \\ 
\vdots \\ 
AAAZ \\ 
AABA \\ 
\vdots \\ 
BAJT \\ 
\vdots \\ 
TSTA \\ 
\vdots \\ 
ZZZZ \\ 
\end{matrix}
$$</p>
<p>$$
\phi^T (x) \phi(z) = K(x, z)
$$</p>
<a class="post-dummy-target" id="参考"></a><h2>参考</h2>
<p><a href="https://katex.org/docs/supported.html" target="_blank">$\KaTeX$ 文档</a></p></div><div class="post-footer" id="post-footer">
    <div class="post-info">
        <div class="post-info-line">
            <div class="post-info-mod">
                <span>本文于 2020-10-17 更新</span>
            </div>
            <div class="post-info-license"></div>
        </div>
        <div class="post-info-line">
            <div class="post-info-md"></div>
            <div class="post-info-share"><span><a href="//www.linkedin.com/shareArticle?url=https%3a%2f%2fdiraclee.gitee.io%2f2020%2f10%2fcs229_lecture8%2f&amp;title=cs229%20%e7%ac%ac%e5%85%ab%e8%8a%82" target="_blank" title="分享到 LinkedIn">
            <i class="fab fa-linkedin fa-fw"></i>
        </a><a href="//service.weibo.com/share/share.php?url=https%3a%2f%2fdiraclee.gitee.io%2f2020%2f10%2fcs229_lecture8%2f&amp;appkey=&amp;title=cs229%20%e7%ac%ac%e5%85%ab%e8%8a%82" target="_blank" title="分享到 Weibo">
            <i class="fab fa-weibo fa-fw"></i>
        </a></span></div>
        </div>
    </div>

    <div class="post-info-more">
        <section><span class="tag">
                        <a href="https://diraclee.gitee.io/tags/%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0/"><i class="fas fa-tag fa-fw"></i>&nbsp;机器学习</a>&nbsp;
                    </span><span class="tag">
                        <a href="https://diraclee.gitee.io/tags/cs229/"><i class="fas fa-tag fa-fw"></i>&nbsp;cs229</a>&nbsp;
                    </span></section>
        <section>
            <span><a href="javascript:window.history.back();">返回</a></span>&nbsp;|&nbsp;<span><a href="https://diraclee.gitee.io">主页</a></span>
        </section>
    </div>

    <div class="post-nav"><a href="https://diraclee.gitee.io/2020/10/cs229_lecture7/" class="prev" rel="prev" title="cs229 第七节"><i class="fas fa-angle-left fa-fw"></i>cs229 第七节</a>
            <a href="https://diraclee.gitee.io/2020/10/sklearn-%E5%B8%B8%E7%94%A8%E6%A8%A1%E5%9E%8B%E5%8F%82%E6%95%B0/" class="next" rel="next" title="机器学习常用模型及其 sklearn 包">机器学习常用模型及其 sklearn 包<i class="fas fa-angle-right fa-fw"></i></a></div>
</div><div class="post-comment"></div>
    </article></div>
            </main><footer class="footer">
    <div class="copyright"><div class="copyright-line">由 <a href="https://gohugo.io/" target="_blank" rel="external nofollow noopener noreffer">Hugo</a> 强力驱动 | 主题 - <a href="https://github.com/dillonzq/LoveIt" target="_blank" rel="external nofollow noopener noreffer">LoveIt<i class="far fa-heart fa-fw"></i></a>
        </div>

        <div class="copyright-line"><i class="far fa-copyright fa-fw"></i><span itemprop="copyrightYear">2020</span><span class="author" itemprop="copyrightHolder">&nbsp;<a href="https://diraclee.gitee.io" target="_blank">Dirac Lee</a></span>&nbsp;|&nbsp;<span class="license"><a rel="license external nofollow noopener noreffer" href="https://creativecommons.org/licenses/by-nc/4.0/" target="_blank">CC BY-NC 4.0</a></span></div>
    </div>
</footer></div><a href="#" class="dynamic-to-top" id="dynamic-to-top" data-scroll>
            <span>&nbsp;</span>
        </a><script src="/js/lib/jquery/jquery.slim.min.js"></script><script src="/js/lib/lazysizes/lazysizes.min.js"></script><script src="/js/lib/smooth-scroll/smooth-scroll.polyfills.min.js"></script><script>window.scroll = new SmoothScroll('[data-scroll]', {speed: 300, speedAsDuration: true});</script><link rel="stylesheet" href="/css/lib/katex/katex.min.css"><script src="/js/lib/katex/katex.min.js"></script><script defer src="/js/lib/katex/auto-render.min.js"></script><link rel="stylesheet" href="/css/lib/katex/copy-tex.min.css"><script defer src="/js/lib/katex/copy-tex.min.js"></script><script defer src="/js/lib/katex/mhchem.min.js"></script><script>
        document.addEventListener("DOMContentLoaded", function () {
            renderMathInElement(document.body, {
                delimiters: [
                    { left: "$$", right: "$$", display: true },
                    { left: "\\(", right: "\\)", display: false },
                    { left: "\\[", right: "\\]", display: true },{ left: "$", right: "$", display: false },]
            });
        });
    </script><script src="/js/blog.min.js"></script></body>
</html>